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1.  SCOPE 

Bit  error  rate  (BER)  bench  testing  of  digital  radios  in  a  back-to-back 
mode  essentially  determines  hardware  characteristics  independent  of  external 
anomalies  such  as  impulse  noise,  fading,  and  interfering  signals.  This  method 
employs  the  statistic  of  a  characteristic  theoretical  curve  of  a  long  term  BER 
versus  received  signal  level  (RSL).  The  receiver  attenuation  is  initially  set 
to  produce  a  BER  of  approximately  10"^  at  a  corresponding  RSL.  A  suitable 
number  of  errors  should  be  counted  at  this  point  to  insure  a  required  level  of 
confidence  (Crow*)  and  to  produce  a  sufficient  accuracy  so  that  the  system's 
theoretical  bit  error  function  curve  can  be  shifted  laterally  to  fit  through 
this  point.  By  taking  an  additional  point  at  10~6  BER,  a  check  can  be  made 
to  assure  that  the  equipment  is  operating  properly.  This  statistical  method¬ 
ology  used  to  test  BER  will  save  testing  time  by  4  or  5  orders  of  magnitude. 
Experimental  and  theoretical  validation  of  this  technique  by  L.C.  Schooley  and 
G.R.  Davis  of  the  University  of  Arizona  is  presented  in  Volume  II  of  their 
report. 2 


1.  Crow,  E.L.,  "Confidence  Limits  for  Digital  Error  Rates",  US  Department  of 
Commerce,  Office  of  Telecom.,  November  1974  (OTP  74-51). 

2.  L.C.  Schooley  and  G.R.  Davis,  Digital  Communications  Systems:  Test  and 
Evaluation  Studies,  Volume  II  (AD  No.  A097123),  Engineering  Experiment 
Station,  College  of  Engineering,  The  Unversity  of  Arizona,  31  August  1979  (Vol 
1,  AD  No.  A097122) . 

Approved  for  public  release;  distribution  unlimited. 
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2.  FACILITIES  AND  INSTRUMENTATION 

2.1  Facilities .  A  radio  test  facility  equipped  with  test  benches  complete 
with  power  sources,  required  tools,  required  test  equipment,  and  accessories 
such  as  connectors,  clips,  and  test  cables. 

2 .2  Instrumentation  and  Equipment 

Item  Requirement/Tolerance 


Data  generator 

Data  error  detector 

Attenuator 

Radio  transmitter,  under  test 
Radio  receiver,  under  test 

2.3  Suggested  Equipment 
Item 

Attenuator 

Data  generator 
Error  detector 

Data  generator/error  detector 
3.  PREPARATION  FOR  TEST 


A  data  generator  capable  of  producing 
the  data  bit  stream  required  by  the 
radio  system  under  test  and  the  bit 
pattern  of  interest. 

Error  detector  must  be  capable  of 
detecting  and  conducting  a  bit-by-bit 
analysis  of  the  transmitted  bit  stream. 

As  required. 


Equipment 

HP382  or  equivalent,  175  dB  maximum 
attenuation. 

HP3760  and  3762. 

HP3761  and  3763 

HP1645,  HP3780,  Aydin  604,  or  equiva¬ 
lent  Data  Test  Sets. 


3.1  Facilities.  Assure  facilities  are  available. 

3.2  Equipment .  Assemble  instrumentation  and  radios  to  conform  with  figure  1, 
(page  5). 

3.3  Instrumentation.  Connect  the  data  generator  to  the  radio  transmitter. 
Connect  the  attenuator  between  the  radio  transmitter  and  the  radio  receiver. 
Connect  the  output  of  the  radio  receiver  to  the  data  error  detector  for  error 
detection  and  recording. 
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3.4.1  Test  Item.  Item  serial  number,  nomenclature,  and  characteristics  as 
required . 

3.4.2  Instrumentation.  Type,  model,  serial  number,  manufacturer,  and 
calibration  data  for  each  instrument. 

3.4.3  Personnel  Data.  Technician’s  name,  grade,  and  MOS/series. 

4.  TEST  CONTROLS 


a.  Set  up  instrumentation  with  the  test  item  in  an  appropriate  test 
facility  (TF). 

b.  Set  up  the  radio  transmitter  and  receiver  and  check  them  out  to  be  in 
proper  operating  condition. 

c.  Adjust  the  data  generator  to  provide  a  digital  bit  pattern  of  appro¬ 
priate  size  and  characteristics. 

d.  Conform  the  analysis  of  the  error  rate  data  points  to  standard  sta¬ 
tistical  guidelines.  (See  Crow  1) 

5 .  PERFORMANCE  TESTS 


a.  Ad just  the  attenuation  in  test  set-up  (fig.  1)  to  a  received  signal 
level  (RSL)  that  produces  a  bit  error  rate  (BER)  of  10~4  iaw  reference  1, 
Appendix  B. 

b.  Vary  the  RSL  to  provide  a  BER  of  10”^.  This  will  give  a  second 
point  on  the  BER  versus  RSL  curve.  The  systems  theoretical  BER  (vs)  RSL  curve 
should  now  fit  through  these  measured  points. 

c.  Adjust  attenuation  of  the  radio  received  signal  level  to  develop  a 
BER  of  10“4  and  then  to  a  setting  to  obtain  a  BER  of  10“6.  The  systems 
RSL  versus  BER  theoretical  curve  (theoretical  curve  is  based  on  the  systems 
modulation  scheme  employed)  should  fit  through  the  two  BER  points  measured. 

The  theoretical  curve  will  probably  have  to  be  shifted  "laterally  an  amount 
fixed  by  the  equipment  quality."  (L.  C.  Schooley  and  G.  R.  Davis*) 


3.  Schooley,  L.C.  and  G.R.  Davis,  "Instrumentation  and  Methodologies  for 
Testing  and  Evaluation  of  Digital  Communications  Systems  and  Equipment," 
Technical  Report,  Engineering  Experiment  Station,  University  of  Arizona, 
Tucson,  Arizona,  15  May  1977.  (AD  B020427) 
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6.  DATA  REDUCTION  AND  PRESENTATION 


a.  Present  the  data  on  a  curve  representing  bit  error  rate  versus 
receive  signal  level. 

(1)  Bit  error  rate  (BER)  along  the  ordinate. 

(2)  Received  signal  level  (RSL)  along  the  abscissa. 

(3)  Bit  error  rate  is  the  total  number  of  errors  detected,  divided 
by  the  total  number  of  bits  received  over  the  same  time  period. 

(4)  Received  signal  level  (RSL)  is  the  level  in  dBm  of  the  received 

signal . 

(5)  Plot  a  curve  of  BER  vs  RSL  as  in  figure  2. 

b.  Extrapolate  from  the  plotted  curve  the  received  signal  level  which 
will  produce  the  desired  or  specified  bit  error  rate. 


Recommended  changes  to  this  publication  should  be  forwarded  to  Commander, 
US  Army  Test  and  Evaluation  Command,  ATTN:  DRSTE-AD-M,  Aberdeen  Proving 
Ground,  MD,  21005.  Technical  information  may  be  obtained  from  the  preparing 
activity:  Commander,  US  Army  Electronic  Proving  Ground,  ATTN:  STEEP-MT-T, 

Ft  Huachuca,  AZ  85613.  Additional  copies  are  available  from  the  Defense 
Technical  Information  Center,  Cameron  Station,  Alexandria,  VA  22314.  This 
document  is  identified  by  the  accession  number  (AD  No.)  printed  on  the  first 
page. 
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Figure  1.  Measurement  of  BER  or  errors  for  bench  test. 
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RECEIVED  SIGNAL  LEVEL  (RSL)  in  dBm  or  S/N 


Figure  2.  Bit  error  rate  versus  received  signal  level. 
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APPENDIX  B 
CHECKLIST 

i 

STANDARD  BIT  ERROR  RATE  (BER)  TESTING 

Facility  is  available.  _ 

Instrumentation  is  calibrated.  _ 

Instrumentation  data  recorded.  _ 

Name,  Grade,  and  MOS  of  person  taking 

data  recorded.  _ 

Security  measures  instituted.  _ 

Thermal  equilibrium  obtained.  _ 

Required  test  data  recorded.  _ 

Feasibility/Compatibility  of  data 

generator/detector  and  radio.  _ 

Data  reduced. 
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APPENDIX  C 
BIT  ERROR  RATE  DATA 

Sample 

dBm  Period(s)  Bit  Number  of  Bit  Error  Rate 

Attenuation  (Seconds) _ Rate(R)  Errors  (E)  (BER  -  E/R*S) 
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APPENDIX  D 

D. 1  Confidence  Limits  for  BER 

The  following  information  has  been  summarized  from  Crow  [1].  Only  those 
details  necessary  to  understand  the  use  of  the  tables  for  the  examples  has 
been  included.  It  is  suggested  that  Crow's  complete  work  be  studied  by  anyone 
who  intends  to  apply  the  method. 

It  is  assumed  that  the  system  under  observation  suffers  errors  which 
occur  independently  with  probability  p.  The  purpose  is  to  determine  as  accu¬ 
rately  as  possible  the  true  BER  (p)  of  a  jsystem  if  c  errors  are  observed  in  a 
sample  of  n  bits.  It  is  well  known  that  p  -  c/n  is  the  single  best  estimate 
and  will  approach  the  true  p  as  n  increases  without  bound.  However  for  finite 
n,  p  is  a  random  variable  and  may  differ  widely  from  the  true  value. 

If  p  remains  constant  during  the  total  number  of  samples  n,  and  if  the 
probability  of  error  in  each  bit  is  independent  of  whether  any  other  errors 
have  occurred,  then  the  probability  of  c  errors  in  n  bits  is  governed  by  the 
binomial  distribution.  It  is  then  possible  to  calculate  confidence  limits  for 
P* 


By  confidence  limits  is  meant:  for_a  given  number  of  errors  c  in  a  sam¬ 
ple  number  of  bits  n,  an  interval  about  p  -  c/n  can  be  determined  within  which 
the  true  value  of  p  lies  with  a  given  percent  of  confidence.  For  example, 
assume  a  sample  of  10^  bits  is  taken  and  8  errors  are  observed,  and  we  wish 

to  be  90  percent  certain  of  the  range  of  the  true  value  of  the  BER.  Referring 

to  table  D. 1  we  find  the  number  8  under  the  column  c;  then  moving  across  to 
the  90  percent  confidence  level  columns  we  find  the  factors  L  *  4.0  and  U  «* 
14.4.  We  can  then  be  90  percent  certain  that  the  true  BER  lies  between  L/n  = 
4/10*>  «*  4  x  10“6  and  U/n  ■  14.4/10^  «  1.44  x  10~^» 

When  the  test  is  terminated  after  a  given  number  of  bits  has  been  re¬ 
ceived  and  then  the  number  of  errors  counted,  the  test  is  termed  binomial  sam¬ 
pling  and  the  confidence  limits  are  determined  as  above.  If  the  test  is  to  be 
terminated  after  a  given  number  of  errors  is  counted  then  the  test  is  termed 
inverse  binomial  sampling.  In  this  case,  since  the  test  is  stopped  on  an  er¬ 
ror,  the  confidence  interval  is  biased.  Crow  has  shown  that  in  this  case  the 
lower  limit  remains  the  same,  however,  the  upper  limit  is  taken  from  c-1.  In 
the  example  above,  if  the  test  were  stopped  after  8  errors  were  counted  and  it 

was  then  determined  that  the  total  number  of  bits  sent  was  10&,  the  lower 

limit  would  be  determined  from  c  »  8,  however  the  upper  limit  would  be  deter¬ 
mined  from  c  -  1  or  7.  The  confidence  limits  would  then  be: 

4.0/106  ■  4  x  10“6  and  13.1/106  -  1.31  x  1(T5. 

D. 2  Determination  of  Length  of  Test 


Figure  D. 1  gives  some  statistical  sample  size  relationships.  The  curves 
are  a  plot  of  the  percent  half-length  of  the  confidence  interval  versus  the 
number  of  errors  for  given  levels  of  confidence.  A  30  percent  half-length 
confidence  interval  would  mean  that  the  interval  would  extend  0.3p  on  either 
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Relative  precision  in  estimating  p  from  large  samples  when 
number  of  successes  is  prescribed  and  c/n  .<  0.1. 
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side  of  p  (for  binomial  sampling;  inverse  binomial  sampling  is  slightly 
biased).  An  example  of  the  use  of  the  curves  might  be:  suppose  it  is  desired 
to  determine  a  50  percent  half-length  interval  with  95  percent  confidence. 

The  curves  show  that  approximately  13  errors  would  have  to  be  counted.  This 
then  is  the  stopping  point  for  an  inverse  binomial  sampling  test.  To  deter¬ 
mine  the  desired  length  of  a  binomial  sampling  test  it  is  necessary  to  make 
preliminary  estimate  of  the  actual  BER.  For  example  if  the  BER  were  suspected 
to  be  approximatly  10“6,  then  th^  number  of  samples  required  for  the  above 
case  would  be  estimated  to  be  n  =  c/p  =  13/10”^  =  13  x  10^  bits. 

D. 3  Use  in  Sequential  Testing 

Crow  suggested  the  possibility  of  proceeding  in  a  sequential  manner  but 
did  not  detail  this  method  of  testing.  (See  reference  1,  Appendix  A.) 

In  Crow's  determination  of  confidence  limits  it  is  assumed  that  the  prob¬ 
abilities  of  error  of  the  first  and  second  kind  are  equal,  i.e.,  =  ,  and 

this  is  usually  appropriate  for  the  communications  testing  problem.  Then  the 
confidence  level  for  the  determined  interval  is  1  -  2  .  For  example  for  a  90 

percent  confidence  level  (  =  =  0.05)  the  probability  that  the  true  BER  is 

less  than  the  lower  limit  which  is  0.05  and  the  probability  that  the  true  BER 

is  greater  than  the  upper  limit  is  also  0.05.  As  will  be  seen,  these  upper 
and  lower  limits  will  be  interpreted  as  our  decision  thresholds  A  and  B. 

Example  1  uses  the  confidence  limits  in  the  following  manner.  After 
104  bits  are  received  we  see  from  the  table  that  if  3  errors  are  counted  the 
80  percent  confidence  limits  are  1.1.  x  10~4  and  5.3  x  10-4.  We  are  then 

90  percent  sure  that  the  BER  is  greater  than  1.1  x  10-4  and  the  item  is 

rejected.  However  if  less  than  3  errors  are  counted  no  conclusion  can  be 
reached.  Three  errors  is  then  our  lower  threshold  B,  but  an  upper  threshold 
cannot  yet  be  determined. 

After  3  x  104  bits  have  been  received  we  can  determine  both  a  lower  and 
upper  bound.  If  6  errors  have  been  counted  the  80  percent  confidence  interval 
is  3.15/3  x  104  =  1.05  x  10~4  and  10.5/3  x  104  =  3.5  x  10-4  and  we  are 
therefore  90  percent  certain  the  BER  is  greater  than  1.05  x  10-4.  On  the 
other  hand,  if  no  errors  have  been  received  the  80  percent  confidence  interval 
is  between  0  and  2.3/3  x  104  =  0.77  x  10-4.  We  can  therefore  be  90  per¬ 
cent  certain  that  the  BER  is  less  than  0.77  x  10~4. 

Examples  2  and  3  utilize  inverse  binomial  sampling,  therefore  the  upper 
and  lower  boundaries  are  determined  from  the  table  in  terms  of  the  total 
number  of  bits  received. 
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Table  D. 1.  Factors  for  Confidence  Limits  for  a  Proportion 
(or  Error  Rate  or  Probability  of  an  Error)  if 
Sample  Size  _>40  amd  Observed  Proportion  >0.1 

L  =  factor  for  lower  limit 
U  =  factor  for  upper  limit 
n  =  prescribed  sample  size 
c  =  observed  number  of  errors  (M). In) 

To  get  limits  to  1-digit  accuracy  (2-digit  for  nM0,000),  simply 
divide  factor  (L  or  U)  by  n. 

To  get  limits  to  2-digit  accuracy  for  40<n<10,000) ,  divide 
L  by  n-(c-l-L)/2  and  U  by  n+(U-c)/2. 


Confidence 


Level 

c 

L 

80% 

U 

L 

90% 

U 

L 

95% 

U 

99% 

L 

U 

0 

.000 

2.30 

.000 

3.00 

.000 

3.7 

.0000 

5.3 

1 

.105 

3.9 

.051 

4.7 

.025 

5.6 

.0050 

7.4 

2 

.53 

5.3 

.36 

6.3 

.242 

7.2 

.103 

9.3 

3 

1.10 

6.7 

.82 

7.8 

.62 

8.8 

.34 

11.0 

4 

1.74 

8.0 

1.37 

9.2 

1.09 

10.2 

.67 

12.6 

5 

2.43 

9.3 

1.97 

10.5 

1.62 

11.7 

1.08 

14.1 

6 

3.15 

10.5 

2.61 

11.8 

2.20 

13.1 

1.54 

15.7 

7 

3.9 

11.8 

3.3 

13.1 

2.81 

14.4 

2.04 

17.1 

8 

4.7 

13.0 

4.0 

14.4 

3.5 

15.8 

2.57 

18.6 

9 

5.4 

14.2 

4.7 

15.7 

4.1 

17.1 

3.13 

20.0 

10 

6.2 

15.4 

5.4 

17.0 

4.8 

18.4 

3.7 

21.4 

11 

7.0 

16.6 

6.2 

18.2 

5.5 

19.7 

4.3 

22.8 

12 

7.8 

17.8 

6.9 

19.4 

6.2 

21.0 

4.9 

24.1 

13 

8.6 

19.0 

7.7 

20.7 

6.9 

22.2 

5.6 

25.5 

14 

9.5 

20.1 

8.5 

21.9 

7.7 

23.5 

6.2 

26.8 

15 

10.3 

21.3 

9.2 

23.1 

8.4 

24.7 

6.9 

28.2 

16 

11.1 

22.5 

10.0 

24.3 

9.1 

26.0 

7.6 

29.5 

17 

12.0 

23.6 

10.8 

25.5 

9.9 

27.2 

8.3 

30.8 

18 

12.8 

24.8 

11.6 

26.7 

10.7 

28.4 

8.9 

32.1 

19 

13.7 

25.9 

12.4 

27.9 

11.4 

29.7 

9.6 

33.4 

20 

14.5 

27.0 

13.3 

29.1 

12.2 

31 

10.4 

35 

21 

15.4 

28.2 

14.1 

30.2 

13.0 

32 

11.1 

36 

22 

16.2 

29.3 

14.9 

31.4 

13.8 

-  33 

11.8 

37 

23 

17.1 

30.5 

15.7 

32.6 

14.6 

35 

12.5 

38 

24 

18.0 

31.6 

16.6 

33.8 

15.4 

36 

13.3 

40 

D-4 


TOP  6-2-570 
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Confidence 

Level 

c  L 

80% 

U 

L 

90% 

U 

L 

95% 

U 

L 

99% 

U 

25 

18.8 

33 

17.4 

35 

16.2 

37 

14.0 

41 

26 

19.7 

34 

18.2 

36 

17.0 

38 

14.7 

42 

27 

20.6 

35 

19.1 

37 

17.8 

39 

15.5 

43 

28 

21.5 

36 

19.9 

38 

18.6 

40 

16.2 

45 

29 

22.3 

37 

20.7 

40 

19.4 

42 

17.0 

46 

30 

23.2 

38 

21.6 

41 

20.2 

43 

17.8 

47 

35 

27.7 

44 

25.9 

46 

24.4 

49 

21.6 

53 

40 

32.1 

49 

30.2 

52 

28.6 

54 

25.6 

59 

45 

36.6 

55 

34.6 

58 

32.8 

60 

29.6 

65 

50 

41.2 

60 

39.0 

63 

37.1 

66 

33.7 

71 

55 

46 

66 

43 

69 

41 

72 

38 

77 

60 

50 

71 

48 

74 

46 

77 

42 

83 

65 

55 

77 

52 

80 

50 

83 

46 

89 

70 

60 

82 

57 

85 

55 

88 

50 

95 

75 

64 

87 

61 

91 

59 

94 

55 

100 

80 

69 

93 

66 

96 

63 

100 

59 

106 

85 

73 

98 

70 

102 

68 

105 

63 

112 

90 

78 

103 

75 

107 

72 

111 

67 

117 

95 

83 

109 

80 

113 

77 

116 

72 

123 

100 

87 

114 

84 

118 

81 

122 

76 

129 

n-s 


